Assessing the Predictive Accuracy of Complex Simulation Models

Predictive accuracy is the sum of two kinds of uncertainty–natural variability and modeling uncertainty. This paper addresses the quantification of predictive accuracy of complex simulation models from two perspectives. First, it recognizes that there is a difference between variability and modeling uncertainty; the former can not be reduced with more test information, while the latter can. We suggest that variability is a natural form of uncertainty that can be quantified with probability theory, but that modeling uncertainty is a form that is better addressed by a theoretical foundation that is not based on random variables, but rather random intervals. We suggest possibility theory as the formalism to address modeling uncertainty. The paper discusses the two different methods, and illustrates the power of their integration to address predictive accuracy with a recent case study involving the crushing load of axially loaded metallic spheres.